Visualization Of Medical Data Research Articles

Enabling reliable usability assessment and comparative analysis of medical software: a comprehensive framework for multimodal biomedical imaging platforms

Enabling reliable usability assessment and comparative analysis of medical software: a comprehensive framework for multimodal biomedical imaging platforms

Feasibility and accuracy of a real-time depth-based markerless navigation method for hologram-guided surgery

BackgroundTwo-dimensional (2D) medical visualization techniques are often insufficient for displaying complex, three-dimensional (3D) anatomical structures. Moreover, the visualization of medical data on a 2D screen during surgery is undesirable, because it requires a surgeon to continuously switch focus. This switching focus problem also results in extensive use of intraoperative radiation to gain additional insights for a 3D configuration. The use of augmented reality (AR) has the potential to overcome these problems, for instance by using markers on target points that are aligned with the AR solution. However, placing markers for a precise virtual overlay are time-costly, always have to be visible within the field of view and disrupt the surgical workflow. In this study, we developed ARCUS, a depth-based, markerless AR navigation system, which overlays 3D virtual elements onto target body parts to overcome the limitations of 2D medical visualization techniques.Methods and resultsIn a phantom study, our markerless ARCUS system was evaluated for accuracy and precision by comparing it to a Quick Response (QR) code-based AR registration method. The evaluation involved measuring the Euclidean distance between target points on a 3D-printed face and their corresponding points on the virtual overlay using a robotic arm for precise measurements. Correlations between the measuring points provided by our markerless system and the actual measuring points on the 3D-print were high, with promising consistent Euclidean distances between the 3D points and the virtual points generated by both our markerless system and the Vuforia QR Code system. We also show two clinical examples of ex vivo case studies on cadaveric human specimens where our markerless ARCUS system could be applicable to.ConclusionThe markerless AR navigation system holds strong potential as a 3D visualization method in clinical settings. While both ARCUS and the Vuforia QR code-based method fell short of meeting the surgical threshold of a 2 mm offset, our markerless system demonstrated promising features such as instant registration, markerless operation, and potential compatibility with non-rigid structures. Its automated virtual overlay onto target body parts offers significant advantages, paving the way for investigations into future clinical use.

Towards a Regulatory Framework for Electronic Medical Record Data Visualization.

Physicians struggle to retrieve data from electronic medical records. We evaluated a digital tool that enhances physician efficiency in retrieving and analyzing patient information for treatment decision-making. Our use case is the care of diabetic patients. Evaluation results showed that healthcare providers who used the i4C (Insights for Care) dashboard experienced greater time efficiency than those who used traditional EMR information retrieval methods. A comprehensive evaluation of the i4C Dashboard confirms its effectiveness in facilitating diabetic care data management, as well as its potential application to a wide range of healthcare scenarios. In order to further maximize its effectiveness on clinical efficiency and patient care, future research should focus on improving its usability and scalability.

3D Modeling for Multimodal Visualization of Medical Data

3D Modeling for Multimodal Visualization of Medical Data

Seeing Your Stories: Visualization for Narrative Medicine.

Importance: Narrative medicine (NM), in which patient stories play a crucial role in their diagnosis and treatment, can potentially support a more holistic approach to patient care than traditional scientific ones. However, there are some challenges in the implementation of narrative medicine, for example, differences in understanding illnesses between physicians and patients and physicians' increased workloads and overloaded schedules. This paper first presents a review to explore previous visualization research for narrative medicine to bridge the gap between visualization researchers and narrative medicine experts and explore further visualization opportunities. Highlights: The review is conducted from 2 perspectives: (a) the contexts and domains in which visualization has been explored for narrative medicine and (b) the forms and solutions applied in these studies. Four applied domains are defined, including understanding patients from narrative records, medical communication, medical conversation training in education, and psychotherapy and emotional wellness enhancement. Conclusions: A future work framework illustrates some opportunities for future research, including groups of specific directions and future points for the 4 domains and 3 technological exploration opportunities (combination of narrative and medical data visualization, task-audience-based visual storytelling, and user-centered interactive visualization). Specifically, 3 directions of future work in medical communication (asynchronous online physician-patient communication, synchronous face-to-face medical conversation, and medical knowledge dissemination) were concluded.

Usability of Medical Data Analysis Tool

Abstract This article deals with the issue of usability of an exploratory data analysis tool in the field of medicine. The text portion contains a description of the methods and the visualization procedure. It analyses the current state on usability, medical data visualization and presents the benefits of visualization tools. The goal of this research was to design and implement a suitable visualization tool for the provided dataset, which is the result of cooperation between Technical university of Košice and East Slovak Institute of Heart and Vascular Diseases (VÚSCH) and presents cardiovascular disease data. The resulting web application aimed to provide potential users with an overview of cardiovascular diseases and the influence of their lifestyle on the assumption of such a disease, and for experts to provide different forms of viewing the given data, as well as to facilitate their work. The application was then tested for usability in two iterations where corrections and improvements were made based on the given feedback. Thanks to this iterative procedure and careful reflection of the user feedback the application received endorsement as a valuable tool by the expert evaluator.

Magic Leap 1 versus Microsoft HoloLens 2 for the Visualization of 3D Content Obtained from Radiological Images.

The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the most popular MR head-mounted displays, Magic Leap 1 and Microsoft HoloLens 2, for the visualization of 3D medical imaging data. We evaluate the functionalities and performance of both devices through a user-study in which surgeons and residents assessed the visualization of 3D computer-generated anatomical models. The digital content is obtained through a dedicated medical imaging suite (Verima imaging suite) developed by the Italian start-up company (Witapp s.r.l.). According to our performance analysis in terms of frame rate, there are no significant differences between the two devices. The surgical staff expressed a clear preference for Magic Leap 1, particularly for the better visualization quality and the ease of interaction with the 3D virtual content. Nonetheless, even though the results of the questionnaire were slightly more positive for Magic Leap 1, the spatial understanding of the 3D anatomical model in terms of depth relations and spatial arrangement was positively evaluated for both devices.

A Medical Image Visualization Technique Assisted with AI-Based Haptic Feedback for Robotic Surgery and Healthcare

A lesson learned during the pandemic is that social distancing saves lives. As it was shown recently, the healthcare industry is structured in a way that cannot protect medical staff from possible infectious diseases, such as COVID-19. Today’s healthcare services seem anachronistic and not convenient for both doctors and patients. Although there have been several advances in recent years, especially in developed countries, the need for a holistic change is imperative. Evidently, future technologies should be introduced in the health sector, where Virtual Reality, Augmented Reality, Artificial Intelligence, and Tactile Internet can have vast applications. Thus, the healthcare industry could take advantage of the great evolution of pervasive computing. In this paper, we point out the challenges from the current visualization techniques and present a novel visualization technique assisted with haptics which is enhanced with artificial intelligent algorithms in order to offer remote patient examination and treatment through robotics. Such an approach provides a more detailed method of medical image data visualization and eliminates the possibility of diseases spreading, while reducing the workload of the medical staff.

Application of Medical Image 3D Visualization Web Platform in Auxiliary Diagnosis and Preoperative Planning

Three-dimensional visualization of medical image data can enable doctors to observe images from more angles and higher dimensions. It is of great significance for doctors to assist in diagnosis and preoperative planning. Most 3D visualization systems are based on desktop applications, which are too dependent on hardware and operating system. This makes it difficult to use across platforms and maintain. Web-based systems tend to have limited capabilities. To this end, we developed a web application, which not only provides DICOM (Digital Imaging and Communications in Medicine) image browsing and annotation functions, but also provides three-dimensional post-processing functions of multiplanar reconstruction, volume rendering, lung parenchyma segmentation and brain MRI (Magnetic Resonance Imaging) analysis. In order to improve the rendering speed, we use the Marching Cube algorithm for 3D reconstruction in the background in an asynchronous way, and save the reconstructed model as glTF (GL Transmission Format). At the same time, Draco compression algorithm is used to optimize the glTF model to achieve more efficient rendering. After performance evaluation, the system reconstructed a CT (Computed Tomography) series of 242 slices and the optimized model was only 6.37mb with a rendering time of less than 2.5s. Three-dimensional visualization of the lung parenchyma clearly shows the volume, location, and shape of pulmonary nodules. The segmentation and reconstruction of different brain tissues can reveal the spatial three-dimensional structure and adjacent relationship of glioma in the brain, which has great application value in auxiliary diagnosis and preoperative planning.

Classification regularized dimensionality reduction improves ultrasound thyroid nodule diagnostic accuracy and inter-observer consistency

ProblemConvolutional Neural Networks (CNNs) for medical image analysis usually only output a probability value, providing no further information about the original image or inter-relationships between different images. Dimensionality Reduction Techniques (DRTs) are used for visualization of high dimensional medical image data, but they are not intended for discriminative classification analysis. AimWe develop an interactive phenotype distribution field visualization system for medical images to accurately reflect the pathological characteristics of lesions and their similarity to assist radiologists in diagnosis and medical research. MethodsWe propose a novel method, Classification Regularized Uniform Manifold Approximation and Projection (UMAP) referred as CReUMAP, combining the advantages of CNN and DRT, to project the extracted feature vector fused with the malignant probability predicted by a CNN to a two-dimensional space, and then apply a spatial segmentation classifier trained on 2614 ultrasound images for prediction of thyroid nodule malignancy and guidance to radiologists. ResultsThe CReUMAP embedding correlates well with the TI-RADS categories of thyroid nodules. The parametric version that embeds external test dataset of 303 images in presence of the training data with known pathological diagnosis improves the benign and malignant nodule diagnostic accuracy (p-value = 0.016) and confidence (p-value = 1.902 × 10−6) of eight radiologists of different experience levels significantly as well as their inter-observer agreements (kappa≥0.75). CReUMAP achieve 90.8% accuracy, 92.1% sensitivity and 88.6% specificity in test set. ConclusionCReUMAP embedding is well correlated with the pathological diagnosis of thyroid nodules, and helps radiologists achieve more accurate, confident and consistent diagnosis. It allows a medical center to generate its locally adapted embedding using an already-trained classification model in an updateable manner on an ever-growing local database as long as the extracted feature vectors and predicted diagnostic probabilities of the correspondent classification model can be outputted.

Graph4Med: a web application and a graph database for visualizing and analyzing medical databases

BackgroundMedical databases normally contain large amounts of data in a variety of forms. Although they grant significant insights into diagnosis and treatment, implementing data exploration into current medical databases is challenging since these are often based on a relational schema and cannot be used to easily extract information for cohort analysis and visualization. As a consequence, valuable information regarding cohort distribution or patient similarity may be missed. With the rapid advancement of biomedical technologies, new forms of data from methods such as Next Generation Sequencing (NGS) or chromosome microarray (array CGH) are constantly being generated; hence it can be expected that the amount and complexity of medical data will rise and bring relational database systems to a limit.DescriptionWe present Graph4Med, a web application that relies on a graph database obtained by transforming a relational database. Graph4Med provides a straightforward visualization and analysis of a selected patient cohort. Our use case is a database of pediatric Acute Lymphoblastic Leukemia (ALL). Along routine patients’ health records it also contains results of latest technologies such as NGS data. We developed a suitable graph data schema to convert the relational data into a graph data structure and store it in Neo4j. We used NeoDash to build a dashboard for querying and displaying patients’ cohort analysis. This way our tool (1) quickly displays the overview of patients’ cohort information such as distributions of gender, age, mutations (fusions), diagnosis; (2) provides mutation (fusion) based similarity search and display in a maneuverable graph; (3) generates an interactive graph of any selected patient and facilitates the identification of interesting patterns among patients.ConclusionWe demonstrate the feasibility and advantages of a graph database for storing and querying medical databases. Our dashboard allows a fast and interactive analysis and visualization of complex medical data. It is especially useful for patients similarity search based on mutations (fusions), of which vast amounts of data have been generated by NGS in recent years. It can discover relationships and patterns in patients cohorts that are normally hard to grasp. Expanding Graph4Med to more medical databases will bring novel insights into diagnostic and research.

Computer-Aided Breast Surgery Framework Using a Markerless Augmented Reality Method.

This study proposes a markerless Augmented Reality (AR) surgical framework for breast lesion removal using a depth sensor and 3D breast Computed Tomography (CT) images. A patient mesh in the real coordinate system is acquired through a patient 3D scan using a depth sensor for registration. The patient mesh on the virtual coordinate system is obtained by contrast-based skin segmentation in 3D mesh generated from breast CT scans. Then, the nipple area is detected based on the gradient in the segmented skin area. The region of interest (ROI) is set based on the detection result to select the vertices in the virtual coordinate system. The mesh on the real and virtual coordinate systems is first aligned by matching the center of mass, and the Iterative Closest Point (ICP) method is applied to perform more precise registration. Experimental results of 20 patients' data showed 98.35 ± 0.71% skin segmentation accuracy in terms of Dice Similarity Coefficient (DSC) value, 2.79 ± 1.54 mm nipple detection error, and 4.69 ± 1.95 mm registration error. Experiments using phantom and patient data also confirmed high accuracy in AR visualization. The proposed method in this study showed that the 3D AR visualization of medical data on the patient's body is possible by using a single depth sensor without having to use markers.

Implementation Details for Controlling Contactless 3D Virtual Endoscopy

In the medical world, with the innovative application of medical informatics, it is possible to enable many aspects of surgeries that were not able to be addressed before. One of these is contactless surgery planning and controlling the visualization of medical data. In our approach to contactless surgery, we adopted a new framework for hand and motion detection based on augmented reality. We developed a contactless interface for a surgeon to control the visualization options in our DICOM (Digital Imaging and Communications in Medicine) viewer platform that uses a stereo camera as a sensor device input that controls hand/finger motions, in contactless mode, and applied it to 3D virtual endoscopy. In this paper, we will present our proposal for defining motion parameters in contactless, incisionless surgeries. We enabled better surgeon’s experience, more precise surgery, real-time feedback, depth motion tracking, and contactless control of visualization, which gives freedom to the surgeon during the surgery. We implemented motion tracking using stereo cameras with depth resolution and precise shutter sensors for depth streaming. Our solution provides contactless control with a range up to 2–3 m that enables the application in the operating room.

A configurable method for clinical quality measurement through electronic health records based on openEHR and CQL

BackgroundOne of the primary obstacles to measure clinical quality is the lack of configurable solutions to make computers understand and compute clinical quality indicators. The paper presents a solution that can help clinical staff develop clinical quality measurement more easily and generate the corresponding data reports and visualization by a configurable method based on openEHR and Clinical Quality Language (CQL).MethodsFirst, expression logic adopted from CQL was combined with openEHR to express clinical quality indicators. Archetype binding provides the clinical information models used in expression logic, terminology binding makes the medical concepts consistent used in clinical quality artifacts and metadata is regarded as the essential component for sharing and management. Then, a systematic approach was put forward to facilitate the development of clinical quality indicators and the generation of corresponding data reports and visualization. Finally, clinical physicians were invited to test our system and give their opinions.ResultsWith the combination of openEHR and CQL, 64 indicators from Centers for Medicare & Medicaid Services (CMS) were expressed for verification and a complicated indicator was shown as an example. 68 indicators from 17 different scenes in the local environment were also expressed and computed in our system. A platform was built to support the development of indicators in a unified way. Also, an execution engine can parse and compute these indicators. Based on a clinical data repository (CDR), indicators were used to generate data reports and visualization and shown in a dashboard.ConclusionOur method is capable of expressing clinical quality indicators formally. With the computer-interpretable indicators, a systematic approach can make it more easily to define clinical indicators and generate medical data reports and visualization, and facilitate the adoption of clinical quality measurements.

Studierfenster: an Open Science Cloud-Based Medical Imaging Analysis Platform

Imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in diagnostics, clinical studies, and treatment planning. Automatic algorithms for image analysis have thus become an invaluable tool in medicine. Examples of this are two- and three-dimensional visualizations, image segmentation, and the registration of all anatomical structure and pathology types. In this context, we introduce Studierfenster (www.studierfenster.at): a free, non-commercial open science client-server framework for (bio-)medical image analysis. Studierfenster offers a wide range of capabilities, including the visualization of medical data (CT, MRI, etc.) in two-dimensional (2D) and three-dimensional (3D) space in common web browsers, such as Google Chrome, Mozilla Firefox, Safari, or Microsoft Edge. Other functionalities are the calculation of medical metrics (dice score and Hausdorff distance), manual slice-by-slice outlining of structures in medical images, manual placing of (anatomical) landmarks in medical imaging data, visualization of medical data in virtual reality (VR), and a facial reconstruction and registration of medical data for augmented reality (AR). More sophisticated features include the automatic cranial implant design with a convolutional neural network (CNN), the inpainting of aortic dissections with a generative adversarial network, and a CNN for automatic aortic landmark detection in CT angiography images. A user study with medical and non-medical experts in medical image analysis was performed, to evaluate the usability and the manual functionalities of Studierfenster. When participants were asked about their overall impression of Studierfenster in an ISO standard (ISO-Norm) questionnaire, a mean of 6.3 out of 7.0 possible points were achieved. The evaluation also provided insights into the results achievable with Studierfenster in practice, by comparing these with two ground truth segmentations performed by a physician of the Medical University of Graz in Austria. In this contribution, we presented an online environment for (bio-)medical image analysis. In doing so, we established a client-server-based architecture, which is able to process medical data, especially 3D volumes. Our online environment is not limited to medical applications for humans. Rather, its underlying concept could be interesting for researchers from other fields, in applying the already existing functionalities or future additional implementations of further image processing applications. An example could be the processing of medical acquisitions like CT or MRI from animals [Clinical Pharmacology & Therapeutics, 84(4):448–456, 68], which get more and more common, as veterinary clinics and centers get more and more equipped with such imaging devices. Furthermore, applications in entirely non-medical research in which images/volumes need to be processed are also thinkable, such as those in optical measuring techniques, astronomy, or archaeology.

Ten Open Challenges in Medical Visualization.

The medical domain has been an inspiring application area in visualization research for many years already, but many open challenges remain. The driving forces of medical visualization research have been strengthened by novel developments, for example, in deep learning, the advent of affordable VR technology, and the need to provide medical visualizations for broader audiences. At IEEE VIS 2020, we hosted an Application Spotlight session to highlight recent medical visualization research topics. With this article, we provide the visualization community with ten such open challenges, primarily focused on challenges related to the visualization of medical imaging data. We first describe the unique nature of medical data in terms of data preparation, access, and standardization. Subsequently, we cover open visualization research challenges related to uncertainty, multimodal and multiscale approaches, and evaluation. Finally, we emphasize challenges related to users focusing on explainable AI, immersive visualization, P4 medicine, and narrative visualization.

Graph-based visualization of sensitive medical data

Graph-based visualization of sensitive medical data

Conturarea unui scenariu optimizat de monitorizare la distanță a sănătății vizând atenuarea presiunii induse de pandemia COVID-19

COVID-19 pandemic has had a major impact on most aspects of everyday life and that has led to a significant faster evolution in the digitalization of medical services. Telemedicine and eHealth continue to provide solutions to mitigate this pandemic's effects, with the aim of helping isolated coronavirus patients, as well as people with chronic conditions. This paper presents how COVID-19 pandemic has been influencing the development of remote healthcare monitoring systems as a result of the need to maintain a remote communication between patient and medical specialist and to provide reliable care. In order to demonstrate how a remote healthcare monitoring scenario can be optimized and personalized according to the needs, specificities and the evolving health status of a patient, the building of the Healthcare Monitoring Component from the ongoing RO-SmartAgeing system is presented. This component, based on a series of medical sensors, connected to an Arduino board that collects and transmits data, through a Wi-Fi network, to the Fog/Edge level, offers both the means to real-time monitor the patient's vital signs and to alert healthcare specialists in case of an abnormal measured value. The gathered information is filtered and then sent to a database in the Cloud allowing further analysis and health data-based applications to be provided. The benefits of this solution that allows the real time visualization of medical data, as well as its storage, processing and subsequent analysis are highlighted and correlated with the specificities induced by COVID-19 pandemic.

Current Status Analysis and Prospects of Domestic and Foreign Medical Data Visualization Research

Current Status Analysis and Prospects of Domestic and Foreign Medical Data Visualization Research

3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases

BackgroundThe purpose of this study was to investigate the potential of a combination of 3D mixed-reality visualization of medical images using CarnaLife Holo (MedApp, Poland) system as a supporting tool for innovative, minimally invasive surgery/irreversible electroporation—IRA, Nano-Knife), microwave ablation (MWA)/for advanced gastrointestinal tumors. Eight liver and pancreatic tumor treatments were performed. In all of the patients undergoing laparoscopy or open surgery volume and margin were estimated by preoperative visualization. In all patients, neoplastic lesions were considered unresectable by standard methods.MethodsPreoperative CT or MRI were transformed into holograms and displayed thanks to the HoloLens 2. During operation, the surgeon’s field of view was augmented with a 3D model of the patient’s relevant structures.ResultsThe intraoperative hologram contributed to better presentation of tumor size and locations, more precise setting of needles used to irreversible electroporation and for determining ablation line in case of liver metastases. Surgeons could easily compare the real patient's anatomy to holographic visualization just before the operations.ConclusionsThe combination of 3D mixed-reality visualization using CarnaLife Holo with IRA, MWA and next systemic treatment (chemotherapy) might be a new way in personalized treatment of advanced cancers.